The Isomorphism Problem for u> -Automatic Trees 



Dietrich Kuske 1 , Jiamou Liu 2 , and Markus Lohrey 2 ' * 

Laboratoire Bordelais de Recherche en Informatique (LaBRI), CNRS and Universite 
Bordeaux I, Bordeaux, France 
2 Universitat Leipzig, Institut fur Informatik, Germany 

kuskeSlabri . f r, liu jiamouSgmail . com, 
lohreyS informatik . uni-leipzig . de 



Abstract. The main result of this paper states that the isomorphism for cj-au- 
tomatic trees of finite height is at least has hard as second-order arithmetic and 
therefore not analytical. This strengthens a recent result by Hjorth, Khoussainov, 
Montalban, and Nies [HKMN08] showing that the isomorphism problem for w- 
automatic structures is not in S\ . Moreover, assuming the continuum hypothesis 
CH, we can show that the isomorphism problem for aj-automatic trees of finite 
height is recursively equivalent with second-order arithmetic. On the way to our 
main results, we show lower and upper bounds for the isomorphism problem for 
^-automatic trees of every finite height: (i) It is decidable (77? -complete, resp,) 
for height 1 (2, resp.), (ii) -hard and in 772 f° r height 3, and (iii) 77 n _ 3 - 
and S n _ 3 -haid and in Yl\ n -/± (assuming CH) for all n > 4. All proofs are 
elementary and do not rely on theorems from set theory. 



1 Introduction 

A graph is computable if its domain is a computable set of natural numbers and the edge 
relation is computable as well. Hence, one can compute effectively in the graph. On the 
other hand, practically all other properties are undecidable for computable graphs (e.g., 
reachability, connectedness, and even the existence of isolated nodes). In particular, the 
isomorphism problem is highly undecidable in the sense that it is complete for S\ (the 
first existential level of the analytical hierarchy [Odi89]); see e.g. [CK06, GK02] for 
further investigations of the isomorphism problem for computable structures. These al- 
gorithmic deficiencies have motivated in computer science the study of more restricted 
classes of finitely presented infinite graphs. For instance, pushdown graphs, equational 
graphs, and prefix recognizable graphs have a decidable monadic second-order theory 
and for the former two the isomorphism problem is known to be decidable [Cou89] (for 
prefix recognizable graphs the status of the isomorphism problem seems to be open). 

Automatic graphs [KN95] are in between prefix recognizable and computable graphs. 
In essence, a graph is automatic if the elements of the universe can be represented as 
strings from a regular language and the edge relation can be recognized by a finite state 
automaton with several heads that proceed synchronously. Automatic graphs (and more 
general, automatic structures) received increasing interest over the last years [BG04, 
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IKR02, KNRS07, KRS05, Rub08]. One of the main motivations for investigating auto- 
matic graphs is that their first-order theories can be decided uniformly (i.e., the input 
is an automatic presentation and a first-order sentence). On the other hand, the isomor- 
phism problem for automatic graphs is E\ -complete [KNRS07] and hence as complex 
as for computable graphs (see [KL10] for the recursion theoretic complexity of some 
more natural properties of automatic graphs). 

In our recent paper [KLL10], we studied the isomorphism problem for restricted 
classes of automatic graphs. Among other results, we proved that (i) the isomorphism 
problem for automatic trees of height at most n > 2 is complete for the level II$ n _ 3 
of the arithmetical hierarchy and (ii) that the isomorphism problem for automatic trees 
of finite height is recursively equivalent to true arithmetic. In this paper, we extend our 
techniques from [KLL10] to uj-automatic trees. The class of cj-automatic structures 
was introduced in [Blu99], it generalizes automatic structures by replacing ordinary 
finite automata by Biichi automata on oj-words. In this way, uncountable graphs can 
be specified. Some recent results on w-automatic structures can be found in [KL08, 
HKMN08, KRB08, KuslO]. On the logical side, many of the positive results for au- 
tomatic structures carry over to aj-automatic structures [Blu99,KRB08]. On the other 
hand, the isomorphism problem of w-automatic structures is more complicated than that 
of automatic structures (which is S\ -complete). Hjorth et al. [HKMN08] constructed 
two w-automatic structures for which the existence of an isomorphism depends on the 
axioms of set theory. Using Schoenfield's absoluteness theorem, they infer that isomor- 
phism of w-automatic structures does not belong to S\. The extension of our elemen- 
tary techniques from [KLL10] to w-automatic trees allows us to show directly (without 
a "detour" through set theory) that the isomorphism problem for w-automatic trees of 
finite height is not analytical (i.e., does not belong to any of the levels S^). For this, 
we prove that the isomorphism problem for w-automatic trees of height n > 4 is hard 
for both levels S„-3 and iZ^_ 3 of the analytical hierarchy (our proof is uniform in 
n). A more precise analysis moreover reveals at which height the complexity jump for 
w-automatic trees occurs: For automatic as well as for w-automatic trees of height 2, 
the isomorphism problem is iT^-complete and hence arithmetical. But the isomorphism 
problem for w-automatic trees of height 3 is hard for 77 -J (and therefore outside of the 
arithmetical hierarchy) while the isomorphism problem for automatic trees of height 3 
is TTg-complete [KLL10]. Our lower bounds for w-automatic trees even hold for the 
smaller class of injectively w-automatic trees. 

We prove our results by reductions from monadic second-order (fragments of) num- 
ber theory. The first step in the proof is a normal form for analytical predicates. The 
basic idea of the reduction then is that a subset X C N can be encoded by an u- 
word wx over {0, 1}, where the i-th symbol is 1 if and only if i e X. The combina- 
tion of this basic observation with our techniques from [KLL10] allows us to encode 
monadic second-order formulas over (N, +, x) by cj-automatic trees of finite height. 
This yields the lower bounds mentioned above. We also give an upper bound for the 
isomorphism problem: for cj-automatic trees of height n, the isomorphism problem be- 
longs to Il\ n _ A . While the lower bound holds in the usual system ZFC of set theory, we 
can prove the upper bound only assuming in addition the continuum hypothesis. The 
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precise recursion theoretic complexity of the isomorphism problem for cj-automatic 
trees remains open, it might depend on the underlying axioms for set theory. 

Related work Results on isomorphism problems for various subclasses of automatic 
structures can be found in [KNRS07,KRS05,KLL10,Rub04]. Some completeness re- 
sults for low levels of the analytical hierarchy for decision problems on infinitary ratio- 
nal relations were shown in [Fin09]. 



2 Preliminaries 

Let N + = {1, 2, 3, . . .}. With x we denote a tuple (xi, . . . , x m ) of variables, whose 
length m does not matter. 

2.1 The analytical hierarchy 

In this paper we follow the definitions of the arithmetical and analytical hierarchy 
from [Odi89]. In order to avoid some technical complications, it is useful to exclude 
in the following, i.e., to consider subsets of N + . In the following, fa ranges over 
unary functions on N + , X,- L over subsets of N + , and u, x, y, z, x i; . . . over elements of 
N+. The class E° C 2 N + is the collection of all sets iCN + of the form 

A = {x e N+ | (N,+, x) |= 3 yi Vy 2 ---Qy n : <p(x,yi, . . . ,y n )}, 

where Q = V (resp. Q = 3) if n is even (resp. odd) and ip is a quantifier-free formula 
over the signature containing + and x . The class 77° is the class of all complements of 
E® sets. The classes E®, 77° (n > 1) make up the arithmetical hierarchy. 

The analytical hierarchy extends the arithmetical hierarchy and is defined analo- 
gously using function quantifiers: The class E^ C 2 N + is the collection of all sets 
4CN + of the form 

A = {x e N+ | (N, +, x) h 3/i V/ 2 • • • Qf n : <p(x, /i, . . . , /„)}, 

where Q — V (resp. Q = 3) if n is even (resp. odd) and ip is a first-order formula over 
the signature containing +, x, and the functions /i, . . . , /„. The class 77^ is the class 
of all complements of E\ sets. The classes E\,H\ (n > 1) make up the analytical 
hierarchy, see Figure 1 for an inclusion diagram. The class of analytical sets 3 is exactly 

Un>l ^n- 

As usual in computability theory, a Godel numbering of all finite objects of interest 
allows to quantify over, say, finite automata as well. We will always assume such a 
numbering without mentioning it explicitly. 

3 Here the notion of analytical sets is defined for sets of natural numbers and is not to be con- 
fused with the analytic sets studied in descriptive set theory [Kec95]. 
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Fig. 1. The analytical hierarchy 



2.2 Biichi automata 

For details on Biichi automata, see [GTW02, PP04, Tho97]. Let T be a finite alpha- 
bet. With r* we denote the set of all finite words over the alphabet r. The set of all 
nonempty finite words is r + . An w-word over r is an infinite sequence w = a\ 0203 • • • 
with di € r. We set w[i] = ai for i G N + . The set of all w-words over r is denoted by 

A (nondeterministic) Biichi automaton is a tuple M = (Q, T, A, I, F), where Q is 
a finite set of states, I, F C Q are resp. the sets of initial and final states, and A C Q x 
r x Q is the transition relation. If P = E n for some alphabet S, then we refer to M as 
an n- dimensional Biichi automaton over E. A run of M on an w-word w = 010203 • • • 
is an w-word r = (gi, m, 92) (92, 02, ?3)(o3, 03, 94) ■ ■ • G such that gi G /. The 
run r is accepting if there exists a final state from F that occurs infinitely often in r. 
The language L(M) C _T" defined by M is the set of all w-words for which there 
exists an accepting run. An w-language L C is regular if there exists a Biichi 
automaton M with L(M) = L. The class of all regular w-languages is effectively 
closed under Boolean operations and projections. 

For w-words W\, . . . , w n G r u , the convolution W\ ® W2 ® ■ ■ ■ ® w n € (r n ) LO is 
defined by 

wi ®w 2 <8> • • • ® w n = (wi[l], . . . ,w„[l])(wi[2], . . . , w„[2])(wi[3], . . . ,to n [3]) 

For w = (u>i, . . . , w n ), we write (8)(«J) for w\ <8> ■ • • <8> w n - 

An n-ary relation i? C [r u ) n is called to-automatic if the w-language (gii? = 
{<g)(w) I w G i?} is regular, i.e., it is accepted by some n-dimensional Biichi au- 
tomaton. We denote with R(M) C (F u )" the relation defined by an n-dimensional 
Biichi-automaton over the alphabet r. 

To also define the convolution of finite words (and of finite words with infinite 
words), we identify a finite word u G r* with the w-word uo u , where o is a new 
symbol. Then, for u, v G T*, w G r u , we write u ® v for the w-word u o" ®wo" and 
u®w (resp. w ®u) for u o w ®w (resp. w uo"). 

In the following we describe some simple operations on Biichi automata that are 
used in this paper. 

- Given two Biichi automata M = (Qo, r, I , A , F ) and Mi = (Qi, T, h, Ai, Fi), 
we use M t±J Mi to denote the automaton obtained by taking the disjoint union of 
Mo and M\. Note that for any word u G -T w , the number of accepting runs of 
M W Mi on u equals the sum of the numbers of accepting runs of M and Mi on 
u. 
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- Let, again, M; = r, Ii,A l7 F{) for i E {0, 1} be two Biichi automata. Then 
the intersection of their languages is accepted by the Biichi automaton 

M = (Q x Qi x {0, 1}, r, I xhx {0}, A, F x Q 1 x {0}), 

where ((po,pi,m), a, (q ,qi,n)) G Z\ if and only if 

• (po,a,q ) S Z\i and (pi,a,gi) £ Zii, and 

• if p m F m then n — m and if p m E _F m then n — 1 — m. 

Hence the runs of M on the w-word u consist of a run of M and of Mi on u. The 
"flag" m E {0, 1} in (poifi, m) signals that the automaton waits for an accepting 
state of M m . As soon as such an accepting state is seen, the flag toggles its value. 
Hence accepting runs of M correspond to pairs of accepting runs of M and of 
Mi . Therefore, the number of accepting runs of M on u equals the product of the 
numbers of accepting runs of M and of Mi on u. This construction is known as 
the flag or Choueka construction (cf. [Cho74, Tho90, PP04]). 

- Let S be an alphabet and M = (Q, r, I, A, F) be a Biichi automaton. We use 

® M to denote the automaton obtained from M by expanding the alphabet to 

e x r-. 

E u ®M = (Q,E x r,I,A',F), 
where A' = {(p,(a,a),q) | (p,a,q) £ A 7 a e E}. Note that L(E U) (g> M) = 

i: w ®L(^). 

2.3 u;-automatic structures 

A signature is a finite set r of relational symbols together with an arity ns € N + for 
every relational symbol Sgt.A r-structure is a tuple .4 = (A, (5'- A )ser), where ^4 is 
a set (the universe of .4) and S- 4 C A" s . When the context is clear, we denote S A with 
S, and we write a E A for a E A. Let £ C 4 2 be an equivalence relation on A. Then 
E is a congruence on .4, if (ui, ui), . . . , (u ns , v ns ) E E and (tti, . . . , u ns ) G 5 imply 
(vi , . . . , u ns ) G 5 for all Set. Then the quotient structure A/E can be defined: 

- The universe of A/E is the set of all ^-equivalence classes [tt] for u G A. 

- The interpretation of S E r is the relation {([ui], [u ns ]) \ (ui, . . . ,u ns ) E S}. 

Definition 2.1. An w-automatic presentation over the signature t is a tuple 

P = (r,M,M^,(M s )ser) 
with the following properties: 

- r is a finite alphabet 

- M is a Biichi automaton over the alphabet r. 

- For every S E t, Ms is an rig-dimensional Biichi automaton over the alphabet F. 

- M= is a 2-dimensional Biichi automaton over the alphabet r such that R{M=) is 
a congruence relation on (L(M), (R(Ms))ser)- 

The r-structure defined by the u-automatic presentation P is the quotient structure 
S{P) = (L(M), (R(Ms))ser)/R(M=) . 
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If R(M=) is the identity relation on _T", then P is called injective. A structure A is 
(injectively) co-automatic if there is an (injectively) w-automatic presentation P with 
A = S(P). In [HKMN08] it was shown that there exist w-automatic structures that are 
not injectively w-automatic. We simplify our statements by saying "given/compute an 
(injectively) w-automatic structure A" for "given/compute an (injectively) cj-automatic 
presentation P of a structure S(P) = A". Automatic structures [KN95] are defined 
analogously to cj-automatic structures, but instead of Biichi automata ordinary finite 
automata over finite words are used. For this, one has to pad shorter strings with the 
padding symbol o when defining the convolution of finite strings. More details on oj- 
automatic structures can be found in [BG04, HKMN08, KRB08]. In particular, a count- 
able structure is cj-automatic if and only if it is automatic [KRB08]. 

Let FO[3 K °,3 2 °] be first-order logic extended by the quantifiers 3 K x . . . (n S 
{H , 2 N °}) saying that there exist exactly n many x satisfying The following theo- 
rem lays out the main motivation for investigating w-automatic structures. 

Theorem 2.2 ( [Blu99, KRB08]). From an co-automatic presentation 

P = (r,M,M^,(M s )ser) 

and a formula ip(x) £ FO[3 N °, 3 2><0 ] in the signature r with nfree variables, one can 
compute a Biichi automaton for the relation 

{(oi,...,a„) £i(M) n \S(P) H<^(N,N,...,[a„])}. 

In particular, the F0[3 N °, 3 2 °] theory of any co-automatic structure A is (uniformly) 
decidable. 

Definition 2.3. Let Kbe a class of uj-automatic presentations. The isomorphism prob- 
lem lso(/C) is the set of pairs (Pi,P 2 ) E JC 2 of co -automatic presentations from JC with 
S(P 1 )^S(P 2 ). 

If Si and S 2 are two structures over the same signature, we write Si WS2 for the disjoint 
union of the two structures. We use S K to denote the disjoint union of k many copies of 
the structure S, where k is any cardinal. 

The disjoint union as well as the countable or uncountable power of an automatic 
structure are effectively automatic, again. In this paper, we will only need this property 
(in a more explicite form) for injectively w-automatic structures. 

Lemma 2.4. Let Pi = (_T, M 4 , ML, (Mg)ser) be injective to- automatic presentations 
of structures Si for i G {1,2}. One can effectively construct injectively to-automatic 
copies of S\ tbl £2, Si", and Si such that 

- The universe of the injectively u-automatic copy S of Si tbl £2 equals L(M r ) U 
L(M ) and the relations are given by S s = R{Ml) U R{M 2 S ) provided L(M 1 ) 
and L(M 2 ) are disjoint. 

- The universe of the injectively to-automatic copy S ofSf° is $* <8> L^M 1 ) where $ 
is a fresh symbol. For ieN, the restriction of S to {$ 1 } <g> L(M 1 ) forms a copy of 

Si- » o 

- The universe of the injectively co-automatic copy S of S 2 is {$1, $2}" ® L(M r ) 
where $1 and $2 are fresh symbols. For w € {$i,$2} w . the restriction of S to 
{w} <8> L(M 1 ) forms a copy of Si. 
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2.4 Trees 



A forest is a partial order F = (V,<) such that for every x G V, the set {y \ y < x} 
of ancestors of x is finite and linearly ordered by <. The level of a node x G V is 
|{y | y < x} G N. The height of F is the supremum of the levels of all nodes in V; 
it may be infinite. Note that a forest of infinite height can be well-founded, i.e., all its 
paths are finite. In this paper we only deal with forests of finite height. For all u G V, 
F(u) denotes the restriction of F to the set {v G V | u < v} of successors of u. We will 
speak of the subtree rooted at u. A tree is a forest that has a minimal element, called the 
root. For a forest F and r not belonging to the domain of F, we denote with r o F the 
tree that results from adding r to F as a new root. The edge relation E of the forest F is 
the set of pairs (u, v) G V 2 such that u is the largest element in {x \ x < v}. Note that 
a forest F = (V, <) of finite height is (injectively) cj-automatic if and only if the graph 
(V, E) (where E is the edge relation of E) is (injectively) cj-automatic, since each of 
these structures is first-order interpretable in the other structure. This does not hold for 
trees of infinite height. For any node u G V, we use E(u) to denote the set of children 
(or immediate successors) of u. 

We use T n (resp. 7^) to denote the class of (injectively) w-automatic presentations 
of trees of height at most n. Note that it is decidable whether a given cj-automatic 
presentation P belongs to T n and T,\, resp., since the class of trees of height at most n 
can be axiomatized in first-order logic. 

3 ^-automatic trees of height 1 and 2 

For w-automatic trees of height 2 we need the following result: 

Theorem 3.1 ( [KRB08]). Let A be an lu '-automatic structure and let (p(xi , . . . , x n , y) 
be a formula of F0[3 N °, 3 2 °]. Then, for all a\, . . . , a n G A, the cardinality of the set 
{b G A | A h <p(ai, ••• ,a n ,b)} belongs to N U {H ,2 N °}. 

Theorem 3.2. The following holds: 

- The isomorphism problem Iso(Ti) for uj-automatic trees of height 1 is decidable. 

- There exists a tree U such that {P G | S(P) = U} is U\-hard. The isomor- 
phism problems lso(72) and lso(75)/ or (injectively) to-automatic trees of height 2 
are LJ^-complete. 

Proof. Two trees of height 1 are isomorphic if and only if they have the same size. By 
Theorem 3.1, the number of elements in an w-automatic tree S(P) with P G T\ is either 
finite, No or 2 N ° and the exact size can be computed using Theorem 2.2 (by checking 
successively validity of the sentences 3 K x : x = x for n G N U {H , 2 N °} 4 ). 

Now, let us take two trees T\ and T 2 of height 2 and let Ei be the edge relation of 
Tj and its root. For i G {1, 2} and a cardinal A let be the cardinality of the set 
of all u G Ei(ri) such that = A. Then T\ = T 2 if and only if = for 

4 Where 3 n x : ip(x) for n € N is shorthand for the obvious first-order formula expressing that 
there are exactly n elements satisfying <p. 
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any cardinal A. Now assume that T\ and T 2 are both w-automatic. By Theorem 3.1, for 
all i G {1, 2} and every u e Eifa) we have \Ei(u)\ G N U {K , 2 N °}. Moreover, again 
by Theorem 3.1, every cardinal k A)1 (A G N U {H , 2 n °}) belongs to N U {K , 2 N °} as 
well. Hence, T\ T 2 if and only if, for all k, A G N U {tt , 2*° }: 

Ti h 3 K x : ((n.a:) £ E A 3 A y : (a:, y) G £) 
if and only if T 2 \= 3 K x : {(r 2 ,x) G E A 3 x y : (x, y) 6 E) . 

By Theorem 2.2, this equivalence is decidable for all n, A. Since it has to hold for all 
k, A, the isomorphism of two cj-automatic trees of height 2 is expressible by a /re- 
statement. Hardness for 77° follows from the corresponding result on automatic trees 
of height 2. □ 



4 A normal form for analytical sets 

To prove our lower bound for the isomorphism problem of w-automatic trees of height 
n > 3, we will use the following normal form of analytical sets. A formula of the 
form x G X or x £ X is called a set constraint. The constructions in the proof of the 
following lemma are standard. 

Proposition 4.1. For every odd (resp. even) n £ N+ and every n\ (resp. E^) relation 
A C W + , there exist polynomials pi, qi G N\x,y,z] and disjunctions if>i (1 < i < £) of 
set constraints (on the set variables X\, . . . , X n and individual variables x, y,~z) such 
that x G A if and only if 

I 

QiXl Q 2 X 2 ■ ■ ■ Q n X n 3yVz: f\pi(x,y,z) ^ qi{x, y,z) V ipi(x, y,z, X u . . . , X n ), 

i=i 

where Q\,Q 2 , ■ ■ ■ ,Q n are alternating quantifiers with Q n = V. 

Proof. For notational simplicity, we present the proof only for the case when n is odd. 
The other case can be proved in a similar way by just adding an existential quantification 
3Xo at the beginning. We will write Z' TO (SC, REC) for the set of JC m -formulas over set 
constraints and recursive predicates, 77 m (SC, REC) is to be understood similarly and 
B£ m (SC, REC) is the set of boolean combinations of formulas from E m (SC, REC). 
With Ck ■ N*^ — > Nfe we will denote some computable bijection. 

Fix an odd number n. It is well known that every 77^ -relation A C N!j_ can be 
written as 

A = {x G N; I V/i 3/ 2 •• • V/„ 3y : P(x, y,h,..., /„)}, (1) 

where P is a recursive predicate relative to the functions /i , . . . , /„ (see [Odi89, p. 378]). 
In other words, there exists an oracle Turing-machine which computes the Boolean 
value P(x, y,f\,..., /„) from input (x, y). The oracle Turing-machine can compute a 
value fi(a) for a previously computed number a G N+ in a single step. Therefore we 
can easily obtain an oracle Turing machine M which halts on input x if and only if 
3y : P(x,y,fi, ...,/„) holds. 
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Following [Odi89], we can replace the function quantifiers in (1) by set quantifiers 
as follows. A function / : N+ — > N+ is encoded by the set {62(2;, y) \ f(x) = y}. Let 
func(X) be the following formula, where X is a set variable: 

func(X) = (Va;, y, z,u,v : 62(2;, y) = u A C 2 {x, z) — v A u,v £ X — »■ y = z) A 
(Vx 3y, z : C 2 (x, y) = z A z £ X) 

Hence, func(X) is a 772 (SC, REC)-formula, which expresses that X encodes a total 
function on N. Then, the set A in (1) can be defined by the formula 

VXi : -nfunc(Xi)V3X 2 : func(X 2 ) A- • • VX n : ^func(X„) VR(x, X u . . . , X n ). (2) 

The predicate R can be derived from the oracle Turing-machine M as follows: Con- 
struct from M a new oracle Turing machine N with oracle sets X\, . . . , X n . If the 
machine M wants to compute the value fi(a), then the machine N starts to enumerate 
all b G N+ until it finds b 6 N+ with C%(a, b) G Xi. Then it continues its computation 
with b for fi(a). Then the predicate R(x, X\, . . . , X n ) expresses that machine N halts 
on input x. 

Fix a computable bijection D : N+ — > Fin(N+), where Fin(N+) is the set of all 
finite subsets of N+. Let in(x, y) be an abbreviation for x £ D{y). This is a computable 
predicate. 

Next, consider the predicate R(x, X\, . . . , X n ). In every run of the machine N on 
input x, the machine N makes only finitely many oracle queries. Hence, the predicate 
R(x, Xi,..., X n ) is equivalent to 

n 

3b 3(si, ...,s n ): S(x, b, (si, . . . , s n )) A f\ Vz < b (\n(z, Sj) o- z £ Xj), 

i=l 

where the predicate 5 is derived from the Turing-machine N as follows: Let T be 
the Turing-machine that on input (x, b, (si, . . . , s„)) behaves as iV, but if AT asks the 
oracle whether z £ Xi, then T first checks whether z < b (if not, then T diverges) and 
then checks, whether in(z, Sj) holds. Then ^(x, 6, (si, . . . , s„)) if and only if T halts 
on input (x, b, (si, . . . , s n )). Hence, the predicate S^x, &, (si, . . . , s„)) is recursively 
enumerable, i.e., can be described by a formula from Z"i(REC, SC). Hence the predicate 
R can be described by a formula from Z^REC, SC). 

Note that the formula from (2) is equivalent with a formula 

VX 1 3X 2 ---VX n :<p{x,X), (3) 

where ip is a Boolean combination of R and formulas of the form func(Xj). Since 
all these formulas belong to i72(REC, SC) U Z^REC, SC), the formula ip belongs to 
BZ^REC, SC) C 77 3 (REC, SC). Hence (3) is equivalent with 

VXi 3X 2 ■ ■ ■ VA„Va 3b Vc : f3 (4) 

where (3 is a boolean combination of recursive predicates and set constraints. 

We can eliminate the quantifier block Va by merging it with MX n : First, we can 
reduce Va to a single quantifier Va. For this, assume that the length of the tuple a is k. 
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Then, Va ■ ■ ■ in (4) can be replaced by Va 3a : Ck (a) = a A ■ ■ ■ . Since Ck (a) = a is 
again recursive and since we can merge 3a3 b into a single block of quantifiers 3b, we 
obtain indeed an equivalent formula of the form 

VX X 3X 2 ■ ■ ■ VX n Va 3b Vc : /3' (5) 

where /?' is a boolean combination of recursive predicates and set constraints. 

Next, we encode the pair (X n , a) by the set {2x \ x G X n } U {2a + 1}. Let a(X) 
be the formula 

a(X) = (Vx, y, x', y' : x = 2x' + 1 A y = 2y' + 1 A x, y G X -)■ x = y) A 
(3x,u : x E X A x = 2u+ 1) 

Hence, a(X) expresses that X contains exactly one odd number. Hence, we obtain a 
formula equivalent to (5) by 

- replacing VX n Va-- - with VX n : ->a(X n ) V 3a, a', a" : a" e X n A a" = a' + 1 A 
a' = 2a A - ■ ■ and 

- replacing every existential quantifier 3bi ■ ■ ■ (resp. universal quantifier Vci ■ ■ ■ ) in 
(5) with 3b t 3b[ : b\ = 2b t A ■ ■ ■ (resp. Vc, : 2a V • • • ), and 

- replacing every sub-formula a 6 X n , bi G X„ or Cj G X n with a' G X n , b\ G X„, 
and c' { G X„, resp.. 

All new quantifiers can be merged with either the block 3b or the block Vc in (5). We 
now have obtained an equivalent formula of the form 

VX 1 3X 2 ---VX n 3bVc: p" (6) 

where /?" is a Boolean combination of recursive predicates and set constraints. 

The block 3b ■ ■ ■ can be replaced by 3b Vb : C\ (b) ^ b V • • • , where t is the length 
of the tuple b. Since Ce (b) ^ b is a computable predicate, this results in an equivalent 
formula of the form 

VXi 3X 2 ---VX n 3bVc: ff" 

where j3"' is a Boolean combination of recursive predicates and set constraints. 

Note that the set of recursive predicates is closed under Boolean combinations and 
that the set of set constraints is closed under negation. This allows to obtain an equiva- 
lent formula of the form 

£ 

VX 1 3X 2 --- VX n 3b Vc : f\ (R t V 

where the Ri are recursive predicates and the ipi are disjunctions of set constraints. 

Since the recursive predicates Ri are co-Diophantine, there are polynomials Pi,qt G 
N[6,c, z] such that Ri(b,c) is equivalent with Vz : pi(b,c,z) ^ qi(b,c,z). Replacing 
Ri in the above formula by this equivalent formula and merging the new universal 
quantifiers Vz with Vc results in a formula as required. □ 

It is known that the first-order quantifier block 3y Vz in Proposition 4. 1 cannot be re- 
placed by a block with only one type of first-order quantifiers, see e.g. [Odi89]. 
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5 uj -automatic trees of height at least 4 



We prove the following theorem for injectively w-automatic trees of height at least 4. 

Theorem 5.1. Let n > 1 and G {£, II}. There exists a tree U n .o of height n + 3 
such that the set {P G 7^ +3 | S(P) = U n .g)} is hard for 0\. Hence, 

- the isomorphism problem lso(7^ +3 ) for the class of injectively uj-automatic trees 
of height n + 3 is hard for both the classes n\ and S\, 

- and the isomorphism problem \so{T l )for the class of injectively uj-automatic trees 
of finite height is not analytical. 

Theorem 5.1 will be derived from the following proposition whose proof occupies Sec- 
tions 5.1 and 5.2. 

Proposition 5.2. Let n > 1. There are trees U[Q] and U[\] of height n + 3 such that 
for any set A C N + that is LJ^ if n is odd and T,\ if n is even, one can compute from 
x G N+ an injectively oj-automatic tree T[x] of height n + 3 with T[x] = U[0] if and 
only if x G A andT[x] = U[l] otherwise. 

Proof of Theorem 5.1 from Proposition 5.2. Let n > 1 be odd. Let A be an arbitrary 
set from 77* and set U n M = U[Q] and U n ,s = U[l}. Then the mapping x i-> P[x] is 
a reduction from A to {P G 7^ +3 | S(P) = U n ,n} and, at the same time, a reduction 
from the Z^-set N+ \ A to {P G 7^ +3 | <S(P) = U n , s }. Since A was chosen arbitrary 
from 77^, the first statement follows for n odd. If n is even, we can proceed similarly 
exchanging the roles of U[Q] and U[V\. 

We now derive the second statement. By the first one, the trees U[0] and U[l] are 
in particular injectively w-automatic and of height n + 3, so let P n and Pi be injective 
w-automatic presentations of these two trees. Then P i-> (P, Pq) is a reduction from 
the set {P G 7^ +3 | S(P) ^ U nM } to lso(7;; +3 ) which is therefore hard for n^ +3 . 
Analogously, this isomorphism problem is hard for S^ l+3 . 

Finally, we prove the third statement. For any n > 1, the set 7^ +3 is decidable (since 
the set of trees of height at most 3 is first-order axiomatizable). With P' , P" G 7^ +3 
arbitrary with S(P') ^ S(P"), the mapping 



(Pi,P 2 ) i-> 



(Pi,P 2 ) ifPi,P 2 e7^ +3 

(P',P") otherwise 



is a reduction from lso(7^ +3 ) to Iso(T')- Hence Iso(T') is hard for all levels S\ and 
therefore not analytical. □ 

The construction of the trees T[x], U[0], and U[l] is uniform in n and the formula 
defining A. Hence the second-order theory of (N, +, x) can be reduced to Un>i{ n } x 
lso(7^) and therefore to the isomorphism problem lso(U„>! T 7 \). 

Corollary 5.3. The second-order theory of (N, +, x) can be reduced to the isomor- 
phism problem lso(lJ nGN+ T„) for the class of all injectively io-automatic trees of finite 
height. 
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We now start to prove Proposition 5.2. Let A be a set that is U\ if n is odd and E\ 
otherwise. By Proposition 4.1 it can be written in the form 

l 

A = {x e N+ | . ..Q n X n 3yVz: f\pi{x,y,z) ^ qt(x,y,z) V ipi(x,y,z,X)} 

i=l 

where 

- Qi, Qii ■ ■ ■ , Qn are alternating quantifiers with Q n = V, 

- Pi, qi (1 < i < £) are polynomials in N[x, y, ~z] where ~z has length k, and 

- every ipi is a disjunction of set constraints on the set variables X\ , . . . , X n and the 
individual variables x, y, z. 

Let </9_i(x, y, X\, . . . , X n ) be the formula 

i 

Vz : f\pi(x,y,z) ^ q i (x,y,z)Vtp l (x,y,z,X) . 

i=l 

For < m < n, we will also consider the formula <^ m (x, X\, . . . , X„_ m ) defined by 

Qn+l-m^n+l-m • • • Q n X n 3y ] <fi-l(x, y, X\, . . . , X n ) 

such that ip (x,Xi, . . . , X n ) is a first-order formula and ¥?„(x) holds if and only if 

x G A. 

To prove Proposition 5.2, we construct by induction on < m < n height-(m + 3) 
trees T m [Xi, . . . ,X n _ m ,x] and J7 m [i] where -X"i, . . . , X n — m C N+, x G N+, and 
i G {0, 1} such that the following holds: 

VX G (2 N +) — Vx G N + : T m [X,x] ~ ^ (7) 

I t/ m [1J otherwise 

Setting T[x] = T„[x], t/[0] = U n [0], and = U n [l] and constructing from x an 
injectively cj-automatic presentation of T[x] then proves Proposition 5.2. 

5.1 Construction of trees 

In the following, we will use the injective polynomial function 

C : -> N + with C(x, y) = (x + y) 2 + 3x + y. (8) 

For ei, e2 G N+, let 5[ei, denote the height-1 tree containing C(ei, e-z) leaves. For 
(X, x, y, z, z k+ i) G (2 N + )™ x N++ 3 and 1 < i < £, define the following height-1 tree, 
where £, pi, and % refer to the definition of the set A above: 5 



T'\X,x,y,z,z k +i,i] 



S[l,2] if ipi(x,y,z,X) 

S\pi(x 7 y,z) + z k+1 ,qi(x,y,z) + z k+1 ] otherwise. 

(9) 



The choice of S[l, 2] in the first case is arbitrary. Any S[a, b] with a =fc b would be acceptable. 
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Next, we define the following height-2 trees, where k E N + U {m} (we consider the 
natural order on N + U {uj} with n < uj for all n E N + ): 

' ' yJ \\+){r[X,x,y 7 z,z k+1 ,i}\zeN k + ,z k+1 EN+,l<i<£} / 



U"[k] = r o (l+J{5[ ei , e 2 ] | e x ± e 2 } W |+J{S[e, e]| K <e< W } 



Note that all the trees T"[X, x, y] and U"[n] are build from trees of the form S[ei, e 2 ]. 
Furthermore, if S[e, e] appears as a building block, then S[e + a,e + a] also appears 
as one for all a E N. In addition, any building block S[e\, e 2 ] appears either infinitely 
often or not at all. In this sense, U"[k] encodes the set of pairs {(ei, e 2 ) | e\ ^ e 2 } U 
{(e, e) | k < e < uj} and T"[X, x, y] encodes the set of pairs {(ei, e 2 ) | ei ^ e 2 }U 
{(p i (x,y,z) + z k+ i,q l (x 7 y,z) + z k +i) | 1 < i < £,x,y,z k +i E N + ,z£ N+}. These 
observations allow to prove the following: 

Lemma 5.4. Let X e (2 N +) n and x, y E N+. Then the following hold: 

(a) T"[X,x,y] ^ U"\n\forsome k E N+ U {uj}_ 

(b) T"[X,x,y] = U"[w] ifandonlyifip-x{x,y,X) holds 

Proof. Let us start with the second property. Suppose tp-i (x, y, X) holds. Let z E N+, 
Zh+i E N, and 1 < i < I. Since Pi(x, y, z) ^ ^(x, 2/,z), there are natural numbers 
d ^ e 2 with T'\X, x, y,z, z k +i,i] = S[e 1 ,e 2 }. Hence T"\X, x, y] S E/"[w]. 

Conversely, suppose T"[X,x,y] = f7 w . Let z E N fe , £fe + i E N, and 1 < i < 
I. Then T'[X,x,y,z,z k+1 ,i] is a height-2 subtree of T"[X,x,y] = U"[lj]. Hence 
there are natural numbers e\ ^ e 2 with T'[X, x, y, z, z k +i, i] = S'[e 1 ,e 2 ]. By (9), 
this implies pi(x,y,z) ^ qi(x,y 7 z) V tjji(x,y,z,X). Hence we showed that Vz : 
/\\=iPi{ x -,Viz) ^ Qi{x,y,z) V ipi(x,y,z,X) holds. _ 

Now it suffices to prove the first statement in case Lp_ x (x, y, X) does not hold. Then 
there exist some ~z E and 1 < i < I with 

Pi(x,y,z) = q l (x,y,z) A -^ipi(x,y,z,X) . 

Hence there is some e E N + such that S[e, e] appears in the definition of T"[X, x, y]. 
Let m = min{e E N+ | S[e,e] appears in T"[X, x, y]}. Then, for all a E N, also 
S[m + a,m + a] appears in T"[X, x,y]. Hence T"[X, x,y] = U"[m\. □ 

In a next step, we collect the trees T"[X, x, y] and U"[k] into the trees T [X, x], Uo[0], 
and Ua[l] as follows: 



T \X, x] - r o (\+){U"[m] | m G N+} W |+J{T"[X, x, y] | y E N+}) 
^o[0] = r o (|+J{[/"[«] | k G N+ U M})*° 
Co[l] =ro (l+J{^"N |meN + }) N ° 



No 
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By Lemma 5.4(a), these trees are build from copies of the trees U" [k] (and are therefore 
of height 3), each appearing either infinitely often or not at all. 



Lemma 5.5. Let X e (2 N +) n and x G N+. Then 

T [X,x] 



U o [0] if ip (x, X) holds and 
U [l] otherwise. 



Proof. If T [X, x] = U [% then there must be some y e N+ such that T"[X, x,y] = 
U"[u>]. By Lemma 5.4(b), this means that tpo(x, X) holds. 

On the other hand, suppose T [X, x] ^ C/ o [0]. Then T"(X, x,y\ ¥ U"[w] for all 
y e N+. From Lemma 5.4(b) again, we obtain for all y G N+: T"[X, x, y] = U"[m y ] 
for some m y e N+. Hence T [X, x] = U$[l] in this case. □ 

Now, we come to the induction step in the construction of our trees. Suppose that for 
some < m < n we have height-(m + 3) trees T m [Xi, . . . , X n _ m ,x], U m [0] and 
U m [l] satisfying (7). Let X stand for (X\, . . . , I n _ m _i) and let a = m mod 2. We 
define the following height-(m + 4) trees: 

T m+1 [X, x]=ro (u m [a\ W |+J {T rn [X, X n _ m , x] \ X n _ m C N+}) 

U m +i[i\ = r o (U m [a] W C/ m W) 2N ° for i e {0, 1} 

Note that the trees T m+ i[X, x], U m+ i[0), and J7 m+ i[l] consist of 2 N ° many copies of 
?/7n[a] and possibly 2 N ° many copies of U m [l — a]. 



Lemma 5.6. Let Xi,..., X„_ m _i C N + and x E N + . Then 
T m +i[Xi, . . . , X„_ m _i, x] 



U m+ i[0] ifip m+ i(x,Xi,. . . X„_ m _i) holds 
C/ m +i[l] otherwise. 



Proof. We have to handle the cases of odd and even m separately and start assuming 
m to be even (i.e., a = 0) such that the outermost quantifier Q n -m of the formula 
if m+1 (x,X 1 , . . . ,X n _ TO _i) is universal. 

Suppose that <p m+ i(Xi, . . . , X„_ m _i, x) holds. Then, by the inductive hypothesis, 
for each X„_ m C N+, T m [Xi, . . . ,X n _ m ,x] = U m [0}. Hence all height-(m + 3) 
subtrees of T m+ \[Xi, . . . , I„_ ra _i,i] are isomorphic to U, n [0] and thus 

T m+1 [X 1 ,...,X n - m - 1 ,x}=roU m [0f° =IWi[0]. 

On the other hand, suppose that -k^„ 1+1 (X 1 , . . . , X n _ m _i, x) holds. Then there exists 
some set X n _ m such that ^Lp m (Xi, . . . ,X n _ m ,x) is true. Hence, by the induction 
hypothesis, 

T m (Xi , . . . , X n — m7 x) = [1] , 

i.e., T m+ i(Xi, . . . , X„_ m _i, x) contains one (and therefore 2 N ° many) height-(m + 3) 
subtrees isomorphic to t/ m [l]. This implies T m+ i(Xi, . . . ,X„_ m _i,x) = [7 m+ i[l] 
since m is even. 

The arguments for m odd are very similar and therefore left to the reader. □ 
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The following lemma follows from Lemma 5.6 with m = n and the fact that ip n (x) 
holds if and only if x G A. 

Lemma 5.7. For all x G N+, we have T n [x] = U n [0] if x <E A and T n [x] = U n [l] 
otherwise. 

5.2 Injective w-automaticity 

Injectively w-automatic presentations of the trees T m [X, x], U m [0], and U m [l] will be 
constructed inductively. Note that the construction of T m+ \ [X, x] involves all the trees 
T m \X, X n - m , x] for X„_ m C N+. Hence we need one single injectively co •-automatic 
presentation for the forest consisting of all these trees. Therefore, we will deal with 
forests. To move from one forest to the next, we will always proceed as follows: add a 
set of new roots and connect them to some of the old roots which results in a directed 
acyclic graph (or dag) and not necessarily in a forest. The next forest will then be the 
unfolding of this dag. 

The height of a dag D is the length (number of edges) of a longest directed path 
in D. We only consider dags of finite height. A root of a dag is a node without incoming 
edges. A dag D = (V, E) can be unfolded into a forest unfold(D) in the usual way: 
Nodes of unfold(D) are directed paths in D that start in a root and the order relation 
is the prefix relation between these paths. For a root v G V of D, we define the tree 
unfold(Z), v) as the restriction of unfold (D) to those paths that start in v. We will make 
use of the following lemma whose proof is based on the immediate observation that the 
set of convolutions of paths in D is again a regular w-language. 

Lemma 5.8. From a given k G N and an injectively cu-automatic presentation for a 
dag D of height at most k, one can construct effectively an injectively ^-automatic 
presentation for unfold(D) such that the roots o/unfold(Z?) coincide with the roots 
of D and unfold(D,r) = (unfold(D)) (r) for any root r. 

Proof. Let D = ( V, E) = S{P), i.e., V is an w-regular language and the binary relation 
E C V x V is w-automatic. The universe for our injectively w-automatic copy of 
unfold(D) is the set L of all convolutions vq <8> v\ <g) V2 ® • • • <8> v m , where vq is a root 
and (vi, Uj+i) G E for all < i < m. Since the dag D has height at most k, we have 
m < k. Since the edge relation of D is oj-automatic and since the set of all roots in 
D is FO-definable and hence w-regular by Theorem2.2, L is indeed an w-regular set. 
Moreover, the edge relation of unfold(D) becomes clearly cj-automatic on L. □ 

For a symbol a and a tuple e = (ei , . . . , efe) G , we write a e for the cj-word 

a ei <g> a e2 ® • • • ® a ek = (a ei o") ® (a e2 o w ) ® • • • ® (a efc o w ) . 

For an w-language L, we write ®k{L) for ®(L k ). The following lemma was shown 
in [KLL10] for finite words instead of w-words. 

Lemma 5.9. Given a non-zero polynomial p(x) G N\x] in k variables, one can effec- 
tively construct a Biichi automaton B[p(x)] over the alphabet {a, o} k with L(B[p(x)}) = 
(8>fe(a + ) such that for all c G N'jL : B[p(x)} has exactly p(c) accepting runs on input a c . 
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Proof. Biichi automata for the polynomials p(x) = 1 and p{x) = Xj are easily build. 
Inductively, let B[p\(x) + P2(x)] be the disjoint union of £>[pi(x)] and£>[p 2 (^)] and let 
B[p\(x) ■ P2<X)] be obtained from £>[pi(x)] and £>[p 2 (X)] by the flag construction. □ 

For X C N+, let wx G {0, 1}* be the characteristic word (i.e., wx [i] = 1 if and only 
if i G X) and, for ~X = (X 1} . . . , X n ) G (2 N +)", write for the convolution of the 
words wxi ■ 

Lemma 5.10. From a given Boolean combination xj){x\, . . . , x m , X\, . . . , X n ) of set 
constraints on set variables X\, . . . , X n and individual variables X\, . . . , x m one can 
construct effectively a deterministic Biichi automaton A^p over the alphabet {0, 1}™ x 
{a, o} m such that for all X\, . . . , X n C N+, c e N™, the following holds: 

wx 1 ® • • ■ ® t«x„ ® a c G Z/(^) ^=^> Xi, . . . , X„) holds. 

Proof. We can assume that ip is a positive Boolean combination, since the w-word 
w n + \x is simply obtained from wx by exchanging the symbols and 1. Then the 
claim is trivial for a single set constraint. Since cj-languages accepted by deterministic 
Biichi automata are effectively closed under intersection and union, the result follows. 

□ 

Lemma 5.11. For 1 < i < £, there exists a Buchi-automaton A% with the following 
property: For all X G (2 N + )", z G N+, and x, y, Zk+i G N + , the number of accepting 
runs of Ai on the word ffly <8> a^ x ' y ' z ' Zk+1 ^ equals 

C(l,2) if i/)i(x,y,z, X) holds 

C(pi(x,y,z) + z k+1 ,qi(x,y,z) + z k+1 ) otherwise. 

Proof. By Lemma 5.9, one can construct a Biichi automaton Bi, which has precisely 
C(pi(x, y,z) + Zk+i,qi(x,y,z) + Zk+i) many accepting runs on the w-word ® 
a (x,y,z,z k +i) _ Secondly, one builds deterministic Biichi automata C% and d accepting a 
word t»y ® a (^,y,z,zk+i) jf an( j on jy jf tne disjunction ipi(x, y,z, X) of set constraints 
is satisfied (not satisfied, resp.) which is possible by Lemma 5.10. 

Let A be the result of applying the flag construction to C{ and Bi. If X G (2 N +)™, 
z G N\, and x, y, z k +i G N + , then the number of accepting runs of A on the word 
® a ( x ^v^ z ^k+i) equals 

{0 if ipi(x,y,z,X) holds 

C(pi(x,y,z) + z k+1 ,q l (x,y,z) + z k+1 ) otherwise. 

Hence the disjoint union of A and C(l, 2) many copies of Cj has the desired properties. 

□ 

Proposition 5.12. There exists an injectively uj-automatic forest %' = (L',E') of 
height 1 such that 

- the set of roots equals {!,...,£} ® ({0, 1}")™ ® (®fe +3 (a+)) U (b+ &+), 
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- for 1 < i < £, X £ (2 N +)™, x, y, z k+1 £ N+ and z £ N%, we have 

H'{i®Wx®a {x ^^^ k + l) ) ^ T'[X,x,y,z,z k+1 ,i] and 

- for ei, e 2 G N + , we /zave 

W'(6(ei^))S5[ei,e 2 ]. 

Proof. Using Lemma 5.9 (with the polynomial p = C(xi,x 2 )) and Lemma 5.11, we 
can construct a Biichi-automaton A accepting {1, . . . ,£} ® ({0, 1})™ ® (® fc+3 (a + )) U 
(6 + <8) such that the number of accepting runs of A on the cj-word u equals 

(i) C(ei,e 2 ) if u = b^^\ 

(ii) C(l, 2) if u = i ® ® a^'*' 2 ^ 1 ) such that ipi(x, y, z, X) holds, and 

(iii) C(pi(x,y,z) + Zk+i,qi{x,y,z) + Zk+i) i£u = i ® ivj^® a < - x,y,z,Zk + 1 '> such that 
tpi(x,y,~z,X) does not hold. 

Let Run.4 denote the set of accepting runs of A. Note that this is a regular w-language 
over the alphabet A of transitions of A. Now the forest %' is defined as follows: 

- Its universe equals L{A) U Run_4. 

- There is an edge (u, v) if and only if v £ Run^ is a accepting run of A on u £ 
L(A). 

It is clear that H' is an injectively w-automatic forest of height 1 with set of roots L{A) 
as required. Note that (i)-(iii) describe the number of leaves of the height- 1 tree rooted 
at u £ L(A). By (i), we therefore get immediately %'(&( ei,e2 ') = S[ei, e 2 ]. Comparing 
the numbers in (ii) and (iii) with the definition of the tree T'\X, x, y, z, Zk+i,i] in (9) 
completes the proof. □ 

From %' = (L', E'), we build an injectively w-automatic dag V as follows: 

- Thedomainof Pis the set (<g>„({0, 1}") <g> a+ ® a+) U b* U ($*®L'). 

- For u, v £ L', the words $ l ® u and P ® v are connected if and only if i = j and 
(u, v) £ E'. In other words, the restriction of V to $* ® V is isomorphic to "H' N ° . 

- For all X £ (2 N +)™, x,y £ N + , the new root w^®a lyX,v ^ is connected to all nodes 
in 

%*® (({l,...,e}®wx®a( x ' v) ® (® fc+ i(a+)))U{6 (ei ' e2) | ei ^ e 2 }) . 

- The new root e is connected to all nodes in $* <g> {fr( e i> e 2) | ei 7^ e 2 }. 

- For all m G N + , the new root b m is connected to all nodes in 

$* ® {b {e ^ I ei ^ e 2 V ei = e 2 > m}. 

It is easily seen that 2? is an injectively w-automatic dag. Let H" = unfold(P) which is 
also injectively cj-automatic by Lemma 5.8. Then, for all X £ (2 N +) n , x, y, m £ N+, 
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we have 



Vl±J{^'( fo(ei ' e2) ) I ei ^ e 2 } J 
propj.12 /|+|{T'[X,x,?/,0,i] |zeN* +1 ,l<i<^}w\ N ° 
r ° VW{^[ei,e 2 ] | ei ^e 2 } ) 

^ T"[X, X ,y] 
%"{e) = so (\${U'(b^^) | ei ^ e 2 }) H ° 

Prop. 5.12 , X N 

= roy ({5[ ei ,e 2 ] | ei ^e 2 }J 



U"\uj] 



H"(b m ) = 6 m o (|+J{H'(6 (ei ' e2) ) | ei ^e 2 V ei = e 2 >m}) N ° 

Prop. 5.12 / vH 

- r ° ( ^{^[ei, e 2 ] | ei 7^ e 2 V d = e 2 > m}J 

^ c/"H 

From "H" = (L", E"), we build an injectively w-automatic dag V as follows: 

- The domain of T> is the set (®„{0, 1}") ® a+ U {e, 6} U ($* ® L"). 

- For u, v E L", the words % l ® u and $ J ® v are connected by an edge if and only if 
i = j and (u, v) e i.e., the restriction of Z> to $* ® L" is isomorphic to H" N ° . 

- For X e (2 N + )", ieN + , connect the new root ® a 1 to all nodes in 



$* ® (toy ® a x ® a+ U &+) . 

Connect the new root e to all nodes in $* ® b*. 
Connect the new root b to all nodes in $* ® b + . 



Then P is an injectively w-automatic dag of height 3 and we set Ho = unfold (2?o)- 
Then, we have the following: 



- The set of roots of "Ho is ((®n({0, 1}")) ® a+) U {e,b}. 

- For all X G (2 N +) n , x eN+we have: 
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H (wx® a x ) ^ro 



= r o 




= T Q [X,x] 



%(e) = r o (\^{U"(b m ) | to G N}) ° 

= ro (|+|{tf"[/c] | K £N + UH}) 
= U Q [0] 



n (b)=ro(\+){n"(b m )\meE + }) 

= ro (l+J{f/"[m] to G N+}) R 
= (7o[l] 



We now construct the forest Hi, Hi, H3, . . . , H n inductively. For < m < n, suppose 
we have obtained an injectively cj-automatic forest H m = (L m ,E m ) as described in 
the lemma. The forest H m +i is constructed as follows, where a — to mod 2: 



- The domain of H m +i is ® n _ m _i({0, 1} W ) a+ U {e, b} U ({$1, $ 2 } w ® I m ). 

- For u, v G £ m and u', u' G {$1, $2}", the words u' ® u and v 1 ®v are connected 
by an edge if and only if u' = v' and (u, v) 6 _E m , i.e., the restriction of V m+ i to 
{$1, $2}" <S> L m is isomorphic to . 

- For all X G (2 N +)™-" 1 - 1 , a; £ N+, connect the new root ® a 21 to all nodes 



from 




- Connect the new root e to all nodes from {$1, $2}" <8> {e, b a }. 

- Connect the new root b to all nodes from {$1, $ 2 } w <8> {b, b a }. 



In this way we obtain the injectively w-automatic forest H m+ i such that: 



- The set of roots of H m +i is ((<8> 



'n—m— 



i({0,l} w ))®a+)U{e,&}. 
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- For X G (2N+)™-™- 1 and ieN + we have: 

U m+1 {wx Ka^roj |+J{77 m (u^ ® w Xn _ m ® a;) | X n _ m C N+} W ^ m (6 Q )) 

Sro( \+){T m [X, X n _ m , x] | X n _ m C N+} W ?7 m [a]J 
— T" TO+ i[X, a;] 

Wm+i(e) = r o (H m (e) W H m (6 a )) 2><0 
= "(C/ m [0] VU m [a}f° 
= U m+1 [0] 

H m+1 (b) = r o (H m (b a ) W H m (b)f° 
^ro(U m [a]ttU m [l]f° 
= EWi[l] 

Hence we proved: 

Lemma 5.13. From each < m < n, one can effectively construct an injectively 
ui-automatic forest H m such that 

- the set of roots ofH m is (<8>„_ m ({0, 1}") ® a + ) U {e, b}, 

- U m {wx®a x ) = T m [X,x}forallX G (2 N +)"- m and a; e N+, 

- "H m (e) = ?7 m [0], and 

- H m (b) = U m [l]. 

Note that T n [x] is the tree in H n rooted at a x . Hence T n [x] is (effectively) an injectively 
cj-automatic tree. Now Lemma 5.7 finishes the proof of Proposition 5.2 and therefore 
of Theorem 5.1. 

6 ^-automatic trees of height 3 

Recall that the isomorphism problem lso(7^ 1 ) is arithmetical by Theorem 3.2 and that 
lso(7J) is not by Theorem 5.1. In this section, we modify the proof of Theorem 5.1 in 
order to show that already lso(7J) is not arithmetical: 

Theorem 6.1. There exists a tree U such that {P E | S(P) = U} is Ill-hard. 
Hence the isomorphism problem \so(7^) for injectively to-automatic trees of height 3 is 
Il\-hard. 

So let A C N + be some set from 77 \. By Proposition 4.1, it can be written as 

t 

A = {x G N+ : VX 3yVz: f\pi{x,y,z) ^ qi(x,y,z)V tpi(x,y,z,X)}, 

i=i 

where pi and qi are polynomials with coefficients in N and ipi is a disjunction of set 
constraints. As in Section 5, let tp_i(x, y, X) denote the subformula starting with Vz, 
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and let ipo(x,X) = My : ip_i(x,y,X). We reuse the trees T'[X,x,y,z,Zh+i,i] of 
height 1. Recall that they are all of the form S[ei , e 2 ] and therefore have an even number 
of leaves (since the range of the polynomial C : — > N + consists of even numbers). 
For e G N+, let S[e] denote the height-1 tree with 2e + 1 leaves. 

Recall that the tree T" [X, x, y] encodes the set of pairs [e\ , e 2 ) € such that e\ ^ 
e 2 or there exist z, z k+1 , and i with e x = Pi(x, y,z) + z k+1 and e 2 = qi(x,y,z) + z k+1 . 
We now modify the construction of this tree such that, in addition, it also encodes the 
setX C N+: 

' ^{T'fX.s.i/.^Zfc+ii] |?eN^Hi &N+,i<i<e}J 

In a similar spirit, we define U[k, X] for X C N + and k G N + U {w}: 

Prr vi /l+J{S[e] |e£X}w|+J{5[ ei ,e 2 ] | ei ^ e2 }w\ N ° 

Vl+J{S[e,e] | K <e<u;} J 

Thenf[X,x,y] = J7[w,Y] if and only if X = Y and T"[X, x, y] ^ U"[uj], i.e., if and 
only if X = y and <^_i (x, y, X) holds by Lemma 5.4(b). Finally, we set 

T[x] = r o ((+){&[«, X] | X C N+, k G N+} W |+J{f [X, x,y] \ X C N+, j/ G N+})*° 

[/ = r o (J+J{?7[k,X] X C N+,k G N+ U{w}}^° . 
Lemma 6.2. Lef x G N+. 77ze« T[x] = U if and only if x G A. 

Proof. Suppose x G A. To prove T[x] = U, it suffices to show that any height-2 
subtree of T[x] is a subtree of U and vice versa. First, let X C N + and y G N+. 
Then, by Lemma 5.4, there exists n G N + U {uj} with T[X, x, y] = U K and therefore 
T[X,x,y] = U[X,k], i.e., T[X,x,y] appears in U. Secondly, let X C N+. From 
x G A we can infer that there exists some y G N + with Vz : t\ i=1 Pi{x,y,x) ^ 
qi(x,y,z) V ipi(x 7 y,z,X). Then Lemma 5.4 implies U u = T[X, x,y] and therefore 
U[X, uj] = T[X, x, y], i.e., U[X, uj] appears in T[x]. Thus, any height-2 subtree of T[x] 
is a subtree of U and vice versa. 

Conversely suppose T[x] = ?7. Let X C N+. Then U[X, uj] appears in U and 
therefore in T[x\. Since U K ^ f/ w for k G N+, there exists some y G N+ with U u = 
T[X, x, y\. From Lemma 5.4 we then get x G A □ 

6.1 Injective u;-automaticity 

We follow closely the procedure for m = from Section 5.2. 

Proposition 6.3. There exists an injectively ui-automatic forest H' = (L', E') of height 
1 such that 
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- the set of roots equals {1,. . .,£} ® {0, 1}" ® (®fc+3(a+)) U (6+ ® 6+) U c+ 

- /or 1 < i < i, X C N+, x, y, G N+ and J g N+, we /zave 

W(i (*.i/.*.**+i)) S T'[X, x, y, z, z k+1 ,i] 

- for ei, e2 G N+, we /zave 

W(6(ei.e"))SS[ei,e 2 ] 

- /or e G N+, we We H'(c e ) = 5[e] 

Proof. Using Lemma 5.9 twice (with the polynomial C(x\,X2) and with the poly- 
nomial 2xi + 1) and Lemma 5.11, we can construct a Biichi-automaton A accepting 
{1, ...,£} ® {0, 1}" ® ((8>fe+3(a + )) U (6+ ® 6+) U c+ such that the number of 
accepting runs of A on the w-word u equals 

(i) C(ei,e 2 )ifu = M ei ' e2 ), 

(ii) 2e + 1 if u = c e , 

(iii) C(l, 2) if u = i ® ® a^'*' 2 ^ 1 ) such that y, z, X) holds, and 

(iv) C(pi(x,y,z) + Zk+i,qi(x,y,z) + z k+1 ) if u = i ® u^® a (x,y,z,z fe+1 ) such ^ 
ipi(x,y,z,X) does not hold. 

The rest of the proof is the same as that of Proposition 5.12. □ 

From %' = (L 1 , E'), we build an injectively cj-automatic dag V as follows: 

- The domain of V is the set ({0, 1}" ® a+ ® a+) U ({0, 1}" ® 6*) U ($* ® Z/). 

- For u,v <E L', the words ® u and $ 3 ® w are connected if and only if i = j and 
(u, v) g E'. In other words, the restriction of V to $* ® V is isomorphic to 'H mo . 

- For all X C N + , a;, y G N + , the new root ® a^' 2 ^ is connected to all nodes in 

$*®(({1, . . . ,£} ® w x ® a< x '»> ® (® fc+1 (a+))) U | ei ^ e 2 } U {c e | e g 

- For all X C N + , the new root ® e is connected to all nodes in 

$* ® {{b (ei ' e ^ | ei ^e 2 }U{c e |eGX}). 

- For all X C N + and m G N+, the new root wx ® 6 m is connected to all nodes in 

$* ® {{b^-^ | ei ^ e 2 V ei = e 2 > to} U {c e | e G X}). 

It is easily seen that V is an injectively w-automatic dag. Let H" = unfold(P) which is 
also injectively w-automatic by Lemma 5.8. Now computations analogous to those on 
page 12 (using Proposition 6.3 instead of Proposition 5.12) yield for all X C N+ and 

x,y,m G N + : 

U"{w x ®a^) = T[X,x,y] 
U"{w x ®e) S U[u,X] 
U"{w x ®b m ) S f/[m,X] 

From "H" = (£", £?"), we build an injectively cj-automatic dag T> as follows: 
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- The domain of V equals a* U $* ® L". 

- For u, v G L", the words $* ® u and $ J ® i; are connected by an edge if and only 
if i = j and (u, v) G E" . Hence the restriction of T> to $* (g> L" is isomorphic to 

- For a; G N + , connect the new root a x to all nodes in 

$* <8> ({0, 1} W <g> b+ U {0, 1} W ® a x ® a+) . 

- Connect the new root e to all nodes in $* (g) {0, 1}" ® 6*. 

Then £> is an injectively w-automatic dag of height 3 and we set Ho = unfold(2? ). 
The set of roots of Ho is a* . Calculations similar to those on page 20 then yield Tin (e) = 
U and %f){a x ) = T[x] for x G N+. Hence, T[x] is (effectively) an injectively cj- 
automatic tree. Now Lemma 6.2 finishes the proof of the first statement of Theorem 6. 1 , 
the second follows immediately. 

Remark 6.4. In our previous paper [KLL10], we used an iterated application of a con- 
struction very similar to the one in this section in order to prove that the isomorphism 
problem for automatic trees of height n > 2 is hard (in fact complete) for level 77271-3 
of the arithmetical hierarchy. This construction allows to handle a VEl-quantifier block, 
while increasing the height of the trees by only 1. Unfortunately we cannot iterate 
the construction of this section for w-automatic trees of height n in order to prove a 
lower bound of the form Tl\ n _ h for n > 3. On the technical level, its Lemma 3.2 
from [KLL10], which does not hold for second-order formulae. 

7 Upper bounds assuming CH 

We denote with CH the continuum hypothesis: Every infinite subset of 2 N has either 
cardinality K or cardinality 2 N °. By seminal work of Cohen and Godel, CH is inde- 
pendent of the axiom system ZFC. 

In the following, we will identify an w-word w G r u with the function w : N+ — > 
r, (and hence with a second-order object) where w(i) = w[i}. We need the following 
lemma: 

Lemma 7.1. From a given Biichi automaton M over an alphabet r one can construct 
an arithmetical predicate 3CCm{u) (where u : N + — > r) such that: u G L(M) if and 
only ifaccM(u) holds. 

Proof. Recall that a Muller automaton is a tuple M = (Q, r, A, I, F), where Q, r, 
A, and I have the same meaning as for Biichi automata but T C 2 Q . The language 
L(M) accepted by M is the set of all w-words u G F" for which there exists a run 
(q- L ,u[l\,q 2 )(q2,u[2],q 3 ) ■ ■ ■ (q x G I) such that {q G Q \ 3*°i = G T. The 
Muller automaton M is deterministic and complete, if |/| = 1 and for all q G Q, a G r 
there exists a unique p G Q such that (q, a, p) G A. 

It is well known that from the given Biichi automaton M one can effectively con- 
struct a deterministic and complete Muller automaton M' = (Q, _T, A, {<?o}, J~) such 
that L(M) = L(M'), see e.g. [PP04,Tho97]. For a given w-word u : N+ — > r and 
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i G N let q(u, i) G Q be the unique state that is reached by M' after reading the length- 
i prefix of u. Note that q(u, i) is computable from i (if u is given as an oracle), hence 
q(u, i) is arithmetically definable. Now, the formula accAf(w) can be defined as follows: 

\J 3x G N+Vy > x f\ (q(u, y) G A A 3z > y : q(u, z) = p) 

a 

Theorem 7.2. Assuming CH, the isomorphism problem lso(7^) belongs to n 2n _ 4 for 
n > 3. 

Proof. Consider trees T t = S(P l ) for P 1 ,P 2 G T n . Define the forest F = (V, E) as 
F = Ti W T 2 For v G V let F(u) = {w £ F : («,tii) £ F} be the set of children 
of u. Let us fix an w-automatic presentation P = (S, M, M=,M E ) for F. Here, M E 
recognizes the edge relation E of F. In the following, for u G L(M) we write F(u) 
for the subtree F([u]r(m=)) rooted in the F-node [u]a(m=) represented by the oj-word 
u. Similarly, we write E(u) for E([u]r(m=))- We will define a F 2n _ 2fc _ 4 -predicate 
isofe(ui, U2), where u\, u 2 G L(M) are on level fc in F. This predicate expresses that 
F(ui) F( U2 ). 

As induction base, let fc = n — 2. Then the trees F{u\) and F(u 2 ) have height at 
most 2. Then, as in the proof of Theorem 3.2, we have F{u\) = F{u 2 ) if and only if 
the following holds for all k, A G N U {H , 2 N °}: 

F ^(3 K xeV:(([u 1 },x)eEA3 x yeV:(x,y)eE^j O 

(Va; G V : ((M,z) G F A 3 A y G V : (x, ? y) G F)^ . 

Note that by Theorem 2.2, one can compute from k, A G N U {Ho, 2 N °} a Biichi au- 
tomaton M K ,\ accepting the set of convolutions of pairs of w-words (u\, u 2 ) satisfying 
the above formula. Hence F{u\) = F{u 2 ) if and only if the following arithmetical 
predicate holds: 

V/t, A G N U {Ho, 2 N °} : acc Mre ,, («i, u 2 ) . 

Now let < k < n — 2. We first introduce a few notations. For a set A, let count(A) 
denote the set of all countable (possibly finite) subsets of A. For k G N U {H } we 
denote with [k] the set {0, . . . , k — 1} (resp. N) in case k G N (k = Ho). For a function 
/ : (A x B) C and a G A let f[a] : B -> C denote the function with f[a}(b) = 



f(a,b). 

On an abstract level, the formula iso fc (ui, u 2 ) is 

(Va; G F(mi) By G E{u 2 ) : \so k+1 {x, y)) A (12) 

(Va; G E(u 2 ) 3y G F(ui) : iso fe+1 (a;, y)) A (13) 

\/X 1 G count(F(wi))VX 2 G count(F(w 2 )) : (14) 

3x, y G Xi UX 2 : -.iso fe+ i (a;, y) V (15) 
3a; G X\ U X 2 3y G (E(u x ) U F(u 2 )) \ (X x U X 2 ) : \so k+1 {x, y) V (16) 

|*i| = |* 2 |. (17) 
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Line (12) and (13) express that the children of m and u 2 realize the same isomorphism 
types of trees of height n — k — 1. The rest of the formula expresses that if a certain 
isomorphism type r of height- (n — k — 1) trees appears countably many times below 
u\ then it appears with the same multiplicity below u 2 and vice versa. Assuming CH 
and the correctness of iso^+i, the formula isOfe(ui, u 2 ) expresses indeed that F(ui) = 
F{u 2 ). 

In the above definition of isOfe(ui, u 2 ) we actually have to fill in some details. The 
countable set X t G count(i?(u i )) C 2 V of children of [iii]#(M=) (which is universally 
quantified in (14)) can be represented as a function f : [\Xi\] x N — > £ such that the 
following holds: 

Vje [\Xi\] :acc ME (Mi®/i[j']) A Vj, I G [\Xi\] : j = I V -acc M ^ (h\j\ ® Ml})- 
Hence, \/Xi G count(.E(uj)) • • • in (14) can be replaced by: 

Vk s gNU {Ho} V/< : [m] xN^£: 

(3j G : -.acc ME (ui ® /i [.?'])) V 

(3j, J G :j/'A acc M . (/*[?'] ® /i[*D) V • • • . 

Next, the formula 3x, y G X\ U Jf 2 : ->iso/; + i (a;, J/) in (15) can be replaced by: 
V 3 ^ [/c,] : -iisOfc+i(/i[j], V 3j G [«i] 3/ G [k 2 ] : -HSOfc + i(/i[j], / 2 [?]). 

»€{1,2} 

Similarly, the formula 3x G XiUJf 2 By G (£(ii 1 )U£(ji 2 ))\(XiUl 2 ) : \so k+1 (x,y) 
in (16) can be replaced by 

\so k +i(fi[j},v) A 

(acc Mi5 (ui <g> v) V acc Mi5 (t*2 <S> u)) A 
V7 G [ki] : -iacc M = (/i [/] <S>v) A 
V/ G [k 2 ] : -■accM=(/2^] ®«) • 

Note that in line (12) and (13) we introduce a new V3 second-order block of quantifiers. 
The same holds for the rest of the formula: We introduce two universal set quanti- 
fiers in (14) followed by the existential quantifier 3v : N — > £ in the above formula. 
Since by induction, isOfe + i is a 7l2 rl _ 2 ^ fe+1 ^_ 4 -statement, it follows that isOfc(wi, u 2 ) is 

a i7 2 1 ™-2fe-4- statement D 
Corollary 5.3 and 7.2 imply: 

Corollary 7.3. Assuming CH, the isomorphism problem for (injectively) co-automatic 
trees of finite height is recursively equivalent to the second-order theory of (N; +, x ). 

Remark 7.4. For the case n = 3 we can avoid the use of CH in Theorem 7.2: Let us 
consider the proof of Theorem 7.2 for n = 3. Then, the binary relation isoi (which 
holds between two oj-words u, v in F if and only if [u] and [v] are on level 1 and 



V 3j G [Ki] 3v : N -> S : 

»€{1,2} 
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F(u) = F(v)) is a 77° -predicate. It follows that this relation is Borel (see e.g. [Kec95] 
for background on Borel sets). Now let u be an w-word on level 1 in F. It follows that 
the set of all w-words v on level 1 with isoi (u, v) is again Borel. Now, every uncountable 
Borel set has cardinality 2 N ° (this holds even for analytic sets [Kec95]). It follows that 
the definition of isoo in the proof of Theorem 7.2 is correct even without assuming CH. 
Hence, lso(73) belongs to 77^ (recall that we proved 77^ -hardness for this problem in 
Section 6), this can be shown in ZFC. 



8 Open problems 



The main open problem concerns upper bounds in case we assume the negation of the 
continuum hypothesis. Assuming -iCH, is the isomorphism problem for (injectively) 
w-automatic trees of height n still analytical? In our paper [KLL10] we also proved that 
the isomorphism problem for automatic linear orders is not arithmetical. This leads to 
the question whether our techniques for w-automatic trees can be also used for proving 
lower bounds on the isomorphism problem for cj-automatic linear orders. More specifi- 
cally, one might ask whether the isomorphism problem for aj-automatic linear orders is 
analytical. A more general question asks for the complexity of the isomorphism prob- 
lem for cj-automatic structures in general. On the face of it, it is an existential third-order 
property (since any isomorphism has to map second-order objects to second-order ob- 
jects). But it is not clear whether it is complete for this class. 
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